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Abstract. Let H CC C" be a domain with smooth boundary, whose Bergman projec- 
tion B maps the Sobolev space H '^{Q,) (continuously) into H ^{Q). We estabhsh two 
smoothing results: (i) the full Sobolev norm H-B/Hfc^ is controlled by derivatives of / 
taken along a single, distinguished direction (of order < fci), and (ii) the projection of a 
conjugate holomorphic function in iF'iyi) is automatically in H''^(JI). There are obvious 
corollaries for when B is globally regular. 



1. Introduction 

In this paper, we consider the range of the Bergman projection, B, associated to a 
smoothly bounded domain O CC C" through the following question: what functions defined 
on $7 does B map to elements of C°°(J7)? Our particular interest is in finding families of 
functions T (;t C~(Q), such that B (J") C C~(J7). 

We present two partial answers to this general question, both under the hypothesis that 
B satisfies Condition R of Bell-Ligocka [3], which says that B (C~(n)) C C°°(n). In the 
first, we show that an function / which has square-integrable derivatives of all orders 
only in a single, distinguished direction is necessarily mapped by i? to a function in C°°{0,). 
The direction is distinguished by being both tangential to bfl, the boundary of fi, and not 
contained in the complex tangent space to bQ (the last property is called complex transversal, 
for short). Note that no smoothness about / is assumed except in this one direction, yet Bf 
is smooth up to the boundary in all directions. This 'partial smoothing' property of B has 
a different character than traditionally studied mapping properties of B, which concentrate 
on whether B preserves various Banach space structures, and was discovered recently by 
the first two authors [l3]. Our first result here generalizes the main theorem in [13], which 
established partial smoothing of B under the hypothesis that the 9-Neumann operator on Q 
is compact. The second result in this paper says that all conjugate holomorphic functions in 
L^(r2) are mapped to C^{Q) by B. This result differs from the first as functions in A^{Q), 
where A^{Q) denotes the space of square-integrable holomorphic functions on Q, need have 
no derivatives in L'^{Q). 

We now state our results more precisely. H^{ft) denotes the standard Sobolev space 
of order k, and H!^{^1) denotes the Sobolev space of order k involving only derivatives in 
direction T, see Definition 12.111 

Theorem 1.1. Let Q CC C" be a smoothly bounded domain and let T be a smooth vector 
field on 0, that is tangential and complex transversal at the boundary. Suppose that there 
exist a pair (ki, k2) € N x N and a constant Ci > such that 

(1-2) \\Bf\\^^<CM\k, V feH'^^n). 
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Then there exists a constant C2 > 0, such that 

(1.3) lis/ii,, <C2||/|U.„T V f€H^'{n). 

Note that in (II. 2|) . automatically ki > k2. We emphasize that (II. 3p is a genuine estimate: 
if the right hand side is finite, i.e., / has T derivatives up to order ki in L^(il), then 
Bf G and the estimate holds. 

Condition R is equivalent to the statement that for each k2 £ N, there exists fci € N 
such that B : H''^{Q) — > H''^{Q). Combined with the Sobolev Lemma to the effect that 
^kLo-^'^i^) ~ C°°{Q), Theorem 11.11 implies in particular the C°°(r2) result described in the 
second paragraph above: 

Corollary 1.4. Assume 0, and T are as in Theorem and that Condition R holds for 
n. Then B maps Hf := r\f^QHf^{9) (continuously) to C~(ll). 

The method we use to prove Theorem 1 1.1 1 is rather general and applicable in other situa- 
tions, i.e., to other operators connected to elliptic PDEs and to other spaces of holomorphic 
functions. 

The proof consists of two distinct steps. The first step is to show that when B satisfies the 
regularity condition (11. 2p . the H^'^ (il) norm of a holomorphic function g can be estimated by 
pairing g, in the ordinary inner product, against holomorphic functions contained in the 
unit ball of H~'^^[Q?) (see Proposition 12.31 below) . This is a special fact about holomorphic 
functions (but it can also be established for solutions to other homogeneous elliptic systems, 
see [5l[20l[l6] and Appendix B in [8]). This special type of duality for holomorphic functions 
originates in the work of Bell [U [6] , and was subsequently extended and refined in [H [71 
[T5| I20I |2]. For our purposes, duality is used to reduce the problem of estimating ||i?/||fc2 
significantly. In order to illustrate how this works, assume for simplicity that ki = k2 = 1- 
Then, for some constant c > 0, 

\\Bf\\i < csup{\{Bf,h)\ : h € A^{n), < l} 

(1.5) =csup{\{f,h)\:heA^{n), ||/i||_i<l}. 

The key point is that the self-adjointness of B in the ordinary inner product, which yields 
the last equality, 'eliminates' B from the right-hand side of (II. 5p . This obviates the need 
to study the commutator of B with differential operators in order to bound the left-hand 
side of (jl.Sp : estimating such commutators is a difficult problem in general since B is an 
abstractly given integral operator. (This effect also occurs when one can use vector fields 
with holomorphic coefficients to control Sobolev norms of Bf, see |jy.) A similar use of 
duality seems applicable to other self-adjoint operators, and also to B itself in other scales 
of Banach spaces besides H^{Q,). 

Let T be the vector field from Theorem ll.il Once (jl.Sp is in hand, the second step (still 
assuming ki = k2 = 1) consists of replacing h, on the right-hand side of (jl.Sp . by the sum 
of the T derivative of a function, T-Li, and of another function, 7^2) both of whose norms 
are uniformly bounded (when ||/i||-i < 1). Since T is tangential, it follows that 

if,h) = {f,THi+n2) = {T*f,'Hi) + {f,H2) 

<c(||r/|| + 11/11), 

where the last inequality follows after noticing that the adjoint T* differs from —T by 
a term bounded in L^. In this very simple way, the full Sobolev norm of Bf of order 1 is 
controlled by the norm of the derivative of / in the special direction T, provided 7^1,7^2 
exist. 

Proving the existence of T^i and 7^2) and similar functions for higher powers of T, is 
conceptually simple, but, for higher powers, somewhat technical. This fact accounts for 
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much of the length of this paper. To simplify matters temporarily suppose that T is of the 
form T = Ti+ L, where both Ti and L are tangential at the boundary, Ti is real, and L is of 
type (1,0). Note that Ti is complex transversal because T is; consequently, if J denotes the 
complex structure map on C", JTi is transversal to the boundary of Q. Let now h G A^(Q) 
with < 1 be given. The goal is to write h = TT-Li + 'H2 where 117^111/^2, ||'H2||l2 are 

bounded by constants depending only on Q. and T. To this end, consider first 21 o (JTi/i), 
where 21 is the operator of 'anti-differentiation along the direction JTi . Then 

(1.6) h{z) =^0 JTih{z)+h{q), 

where q = q{z) varies in a fixed compact subset of Q. Because h is holomorphic, the 
contributions given by h{q) are easily shown to be bounded in and are folded into the 
function 7i2- Furthermore, the Cauchy-Riemann equations show that JTi{h) = —iTi{h). 
Thus 

2to JTi(/i) = 2lo(-i)ri/i 

(1.7) = -i (21 oT(/i) -21 (L/i)) . 

But Lh vanishes since h is holomorphic. As a last step, we commute T with 21 in ()1.7p : 

(1.8) 2loT(/i) =To2l(/i) + [2l,T] (/i). 

The commutator in (jl.Sp is straightforward to analyze since both 21 and T are explicit 
operators; this term forms the final component of the function 7^2- The term — z2l(/i) is 
the function Tii. The needed inequality ||2t(/i)|| < C||/i||_i is a consequence of Hardy's 
inequality, which says that 21 gains a factor of the boundary distance d{z), together with 
the fact that such a factor gains a derivative in the case of holomorphic functions: 

(1.9) mh)\\ < ci\\d{z)h\\ <C2\\h\U . 



Coming to our second result, we start with an observation about the Bergman projection 
on the unit disk. Modulo constants, conjugate holomorphic functions on the unit disc are 
orthogonal to the Bergman space. Equivalently: their Bergman projections are constant 
functions. Of course, this fails on general (even planar) domains. Our second result says 
that nevertheless, if the Bergman projection satisfies a regularity estimate such as (II. 2p . 
projections of conjugate holomorphic functions are still as good as projections of functions 
smooth up to the boundary: they belong to H^^{^). In particular, if Condition R holds, 
these projections are themselves smooth up to the boundary. 

Theorem 1.10. Let CC C" be a smoothly bounded domain. Suppose that the Bergman 
projection B on VI satisfies the regularity condition (jl.2p . Then there is a constant C > 
such that 

(1-11) \\Bfh,<C\\f\\ 

for all conjugate holomorphic functions f in L^(J7). 

In contrast to Theorem I l.H smoothing here takes place in all directions. But just as with 
Theorem ll.il there is an immediate corollary for when Condition R holds. 

Corollary 1.12. Let 0, be as in Theorem \1.10\ and assume that Condition R holds. Then 
for every /c € N there is a constant > such that 

(1.13) \\Bf\\k<Ck\\f\\ 

for f conjugate holomorphic in L'^(Q). In particular, Bf is smooth up to the boundary. 
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The proof of Theorem 11.101 say again for ki = k2 = 1, also starts with (jl.Sp . But 
now, estimating \{f,h)\ means we are estimating the (absolute value of) the integral of a 
holomorphic function (namely fh) over Q. Such an integral can be dominated by 
for any m € N ([U El [T5| I20j). Finally, the equivalence, for holomorphic functions, of 
membership in a negative Sobolev space and polynomial boundedness in the reciprocal of 
the boundary distance (O |20]) gives the estimate < Cm.^H/H ll^ll-fc for m big 

enough (relative to k). We remark that the last two steps are valid for functions in the 
kernel of more general elliptic operators (systems), see [IT] and |18] . respectively, for the 
relevant results. However, the first step after invoking (jl.Sp may fail: for example, the 
product of two harmonic functions need not be harmonic. 

Regularity properties like p.2p are known to hold on a large class of domains. When 
the domain is pseudoconvex, these properties are essentially equivalent to corresponding 
regularity properties of the 5-Neumann operator. For example when the domain is of finite 
type, or when it admits a defining function that is plurisubharmonic at boundary points 
(in particular, when the domain is convex), (jl.2p holds for the pair (fc,fc) for all /c G N. 
Nonpseudoconvex domains on which regularity estimates for the Bergman projection hold 
include Reinhardt domains, complete Hartogs domains in C^, and domains with 'approxi- 
mate symmetries'. For these results and for further information on the Sobolev regularity 
theory of the Bergman projection and of the 5-Neumann operator, we refer the reader to 
|10l [9l [21] and their references. [9] also contains a discussion of the connection between the 
regularity theory of the Bergman projection and duality of holomorphic function spaces. 

The paper is laid out as follows. In Section [2] we define the notions needed for Theorem 
11.11 collect some standard definitions, and give a new proof of (a portion of) Bell's duality 
theorem on holomorphic functions in Sobolev spaces. In Section [3l we state the anti- 
differentiation result. Proposition 13.11 then give a proof of Theorem 11.11 modulo a proof of 
Proposition 13.11 Section [4] is devoted to a proof of the anti-differentiation result, broken 
down into several subsections. We develop the algebra of operators associated to anti- 
differentiation along integral curves to transverse vector fields in these subsections in some 
detail, as we need these results for our proof of Proposition I3.lt we also mention that the 
results in Section U] can be applied elsewhere. In Section [5] we prove Theorem I l.lOi 



2. Function spaces, duality, and complex transversality 

Throughout the paper, Q will denote a domain with smooth boundary bQ, contained in 
C". The standard inner product and norm on functions defined on U will be denoted 



if, 9) = 




1/2 



where the integrals are taken with respect to the Euclidean volume element. If /c is a positive 
integer, we let H^{Q) denote the usual Sobolev space of complex- valued functions whose 
norm ||.||fc is induced by the inner product 

\a\<k 

for a a multi-index and denoting differentiation of order a. Let Cq°(0) denote the set 
of smooth functions with compact support in and C°°{Q) the set of functions smooth up 
to bQ. As is well-known, C°°(n) is dense in H''{n). We let H^{n) be the closure of C^{n) 
in H''{n). 
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The dual space of H^{n) will be denoted Because C^{n) is dense in H^{n), 

H~'^{Q) imbeds naturally into the space of distributions = {Cq°{Q))* on J7, and 

L G 'D'{^) belongs to H^^{i}) precisely when 

(2.1) \\L\U = sup{|(L,0)| : <A G C^{n), Uh < 1} 

is finite. Here (L, 0) denotes the action of the distribution L on the test function (j). We 
shall only need to compute (j2.ip on certain L € -L^(J7), in which case {L, cj)) = (L, i^). 

The subspace of H^{^) consisting of holomorphic functions will be denoted A^(0). For 
consistency of notation, we let ^{^l) denote the Bergman space: ^{Q.) = L^(r2) n 0{VL), 
where 0{yi) denotes the space of holomorphic functions on i7. For functions / in A~^{Vl), 
the norm is comparable to a weighted norms of /, with weight equal to the 

corresponding positive power of the distance to 617 ([HI [H])- If r is a smooth defining 
function for 17 - i.e., = {r < 0} and dr ^ when r = - we shall use the following 
version of (half of) this fact: for /c € N there exists a constant (3^ > such that 

(2.2) \\{-r)''h\\ < MH-k V/iEA0(J7). 

The Bergman projection is the orthogonal projection of L^(0) onto A^{i^). We say that 
B satisfies Condition R (see [3]) if it maps C°°(0) to itself (by continuity in L^(r2) and the 
closed graph theorem for Frechet spaces, this map is then automatically continuous). This 
property is often also referred to as global regularity. An equivalent formulation is: for each 
/c2 G N there exists /ci € N such that B maps H''^{Q) (continuously) into H''^{Q). The 
special case where (jl.2p holds with ki = k2 is usually referred to as exact regularity at level 
k = ki = k2- We remark that, rather intriguingly, no instance is known where B satisfies 
Condition R, but is not exactly regular at all levels. 

As mentioned in the introduction, a crucial fact for our proof of Theorem 11.11 is that 
regularity of the Bergman projection as in (jl.2p implies that the Sobolev norm of a function 
/ G (^7) is controlled by pairing / with a family of holomorphic functions in the ordinary 
!?■ inner product. Duality results of this kind are known, [H [6l O [20l [71 [151 [2]) but they 
are formulated for exact regularity, rather than (jl.2p . or for Condition R (i.e., assuming 
(|1.2p holds for all k2 G N). In these situations, a clean cut formulation is possible: for 
example, A^(p.) and A~^{Q), the closure of A^{^) in A~^{^), are mutually dual, under a 
natural extension of the L? pairing, when B is exactly regular at level k. When there is a 
loss of derivatives, that is when k2 < ki in (II. 2p . this duality has a somewhat less striking 
formulation. For this reason, we only state below what we need and give, for the reader's 
convenience, a straightforward proof (which seems to be new even for the case ki = ^2)- 

Proposition 2.3. Let Q C C" be a domain with smooth boundary. Suppose that (|1.2p 
holds. Then there exists a constant c > such that 

(2.4) ll/IU, < csup{|(/,/i)| : h G A'^^^Q), \\h\U, < l} 

holds for all f G A^{Q,). The constant c depends on Ci from (II. 2p . n and k2. 

Remark 2.5. It is part of (12. 4p that if the right hand side is finite, then so is the left hand 
side. That is, (12. 4p is a genuine estimate, as opposed to an a priori estimate. 

Proof. Let / G A^{Q) C C°°(ri) and let a be a fixed multi-index with jal < k2. Then 
Z)"/ G L2(J7) if and only if 

sup{|(I?"/,5)| : 9 G C^m, \\g\\ < 1} < 00 . 
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Moreover, in this case, IID^/H is given by this supremum. Because g G C'^{Vt) and / G 
C°°(r2), integration by parts yields 

= |(-1)H (/,Z?"5)| = \{Bf,D'^g)\ = \{f,BD^g)\ . 

Thus, 

(2.6) \\D''f\\ = sup{\{f,BD''g)\:geC^in), \\g\\ < 1} . 

Note that ([O]) imphes that BD^'g G ^^^'2(^1). The aim now is to see that (L2\i forces BD°'g 
to be uniformly bounded in H~^^{^). For that, let 99 G Cq°{Q,) with Hv'llfci < 1- Integration 
by parts and Cauchy-Schwarz give 

(2.7) \{y^,BD''g)\ = \{D^Bip,g)\ < \\B^\\kM\ < CiMlM < Ci , 

where (jl.2p was used again. Thus, (j2.ip shows < Ci. Returning to ()2.6p . we 

now obtain that 



|Z)-/|| = C7isup 



f,BD^-^ 



:g€C^in), \\g\\<l 



<Cisnp[\{f,h)\:heA''^{n), <l}. 



Summing over \a\ < k2 gives (|2.4p with c = Xljlo '-' 
Remark 2.8. Proposition 12.31 implies that when (jl.2p holds, then 

< csup{|(i?/,/i)| : /i G Af'-^in), \\h\U, < 1} 

(2.9) =csup[\{f,h)\:heA''^{n), \\h\U, < l} 

for all / G H^^{Q,). It is (j2.9p that will be used in the proof of Theorem ll.il (compare (jl.Sp 
above). 

Let O CC C" = {p < 0} be a smoothly bounded domain and denote by -L„ := 
Yl^=ii^P/'^){'^/'^^j) the complex normal field of type (1,0). Set To = «(L„ — L™). 
Then Tq is real and tangential to bQ, and it spans (over C) the orthogonal complement 
of ri'0(6ri) e T^''^{bn) m T{bn) ® C. in particular, a vector field T, with coefficients in 
C°°(r2), that is tangential to b^l can be written as 

T = aTo + Y^ + Y2, 

where a is a smooth complex-valued function, and Yi, Y2 are of type (1, 0) and tangential to 
bQ. The vector field T is called complex transversal if it is transversal to T^'^{b^)(BT^'^ (^^)) 
i.e., if a is nowhere vanishing on bil. Writing I2 as a (^{l/a)Y2 + {l/a)Y2) — {a/a)Y2, shows 
that near T can be written in the form 

(2.10) T = a{Ti + L), 

where Ti is real and complex transversal, L is of type (1,0), and both Ti and L are tangential 

to bn. 

The Hilbert spaces H^{n) that occur in Theorem 11.11 are the usual Sobolev spaces with 
respect to differentiation in the direction T^, where the latter is the /c-fold differentiation 
with respect to T: T^{f) = T (T {. . . [Tf) ...)). 

Definition 2.11. For A; G N, set 

H^{n) = {/ G L\n) : T^f G L\n), j G {1, . . . , k}} , 
where f is taken in the sense of distributions. 
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For fixed k, H^{n) is a Hilbert space with respect to the inner product 



if, 9) 



j=0 



and C°°(i7) is dense in ff^(il) with respect to the norm 



2 

k,T 



k 

El 



induced by this inner product. The completeness of H^{ft) is proved in the same way as for 
ordinary Sobolev spaces; density of C°°(r2) is a standard application of Friedrichs' Lemma 
(see for example [TO], Lemma D.l and Corollary D.2). The spaces H^{Q.), k > I, depend 
on the choice of the tangential, complex transversal vector field T; cf. Section 5 in jl3] . 
However, if 6 is a smooth, non-vanishing, complex-valued function, then H^{Q,) and H^j,{Q) 
are equal, and an inductive argument gives, e.g., the somewhat rough estimate 



(2.12) 



{hTff <hk\\f\\k,T for hk 



_max 

Q.fi<l<k 



1 



This implies in particular that Theorem 11.21 holds for T if and only if it holds for hT. 
Moreover, (j2.12p indicates how the constant in (jl.Sp changes (although there are other 
quantities that determine the constant in (11.31) ). 

The Prechet space H^{^) = n'^QH^{Q) is equipped with the (metrizable) topology 
induced by the family of norms {||.||a;,t : k G N} (see e.g. [19]). It inherits completeness 
from the H^{Q). For a class of examples of the fact that C°°{Q) C H^{Q,), see Section 5 
in [13]. 



3. Proof of Theorem 11.11 

As indicated in the introduction, the second step in proving Theorem 11.11 relies on a 
representation of a holomorphic function h in terms of T"' -derivatives, j < k, of functions 
whose norms are controlled by for k gN. 

Proposition 3.1. Let Q CC C" be a smoothly bounded domain and let T be a vector field 
which is tangential to bfl and complex transversal. Let G N. 

There exists an open neighborhood U of bQ, a function G CQ°{i} fl U) which equals 1 
near bCl, and constants Cj^k > for j G {0, . . . , k}, such that for all h G A^{Q) there exist 
functions Tij G H''{^}) n C°°{Q), j G {0, . . . , k}, on satisfying 

(i) C/i = E'=o^X' 

(ii) W'H'JW < Cj,k\\h\\^k for all j e {0, . . . ,k}. 

Remark 3.2. For simplicity, we have only stated what we need in Proposition [3Tj As usual 
in such situations, membership in H^{ft) actually comes with a norm estimate that can be 
worked out with a little additional care. In fact, the arguments in ^4.41 can be refined to 
show that for some constant Cs^k > 0, ||?^^(/i)||s < Cs^fc||^||s-fc, for s G M. 

The proof of Proposition 13.11 will be given in Section HI Assuming it is true, we now give 
the proof of Theorem II. 1[ We will use the notation A < B to denote the existence of a 
constant C independent of /, but allowed to depend on O and T, such that A < C ■ B. 
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Proof of Theorem Assume first f ^ 
(1231) states that 



Then Bf € A^^{n), by (USD. InequaHty 



(3.3) 



<csup{|(/,/i)l :/ie^^^(0), \\h\U, < l} 



holds for some constant c > independent of /. 

We use Proposition 13. II for k = ki to estimate the right-hand side of (|3.3p as follows. For 
T as in Theorem lL2l choose the neighborhood U of bVl and the cut-off function C, described 
in Proposition 13. II Using the partition of unity 1 — C} yields 

(3.4) \{fM < |(/,C/i)| + I(/,(l-C)/i)| 

ki 

= \{f,J2T'H'^)\ + \{f,{l-0h)\. 

j=0 

As (1 — C) is identically zero near bil., it follows from the Cauchy-Schwarz inequality, (|2.2p . 
and ||/i||-A:i < 1 that 



(3.5) 



|(/,(i-C)/i)l < A 



\h\\-ki < All/ll, 



where the constant /3fc > depends on (see (j2.2p ) and C • Since T is tangential, integration 
by parts (justified since gives 



Tf,T^ 'n''A + (gof,T' 



where go, depending on the first order derivatives of the coefficients of T, is a smooth 
function on $7. Continuing this procedure yields 



{f,T'n';^)\<j2\\9iT'f 



£=0 



for functions gi G C°°{Q)^ H. G {0, . . . , j — 1} and gj = 1. By part (ii) of Proposition 13.11 it 
now follows that 



Summing over j < ki gives, in view of (|3.3p . (|3.4p . and (13. 5p that 

(3.6) \\Bf\\,^ < C\\fU,,T V / G C^m. 

To remove the smoothness assumption on /, approximate / with respect to ||.||fci.r by 
functions in C°°(r2). Then invoke ()3.6p and the continuity of B in L^{Q) to obtain (II. 3p : 
for more details see Lemma 4.2 in Il3l. □ 



4. Anti-differentiation along a transverse direction 

4.1. General set-up. In this and the next two sections we work in the real setting of 
M". Let Q he a bounded domain with smooth boundary. Denote by A/" = Yl^=i-^j'^ ^ 
vector field with smooth coefficients in a neighborhood of fi. Suppose that is transversal 
to feQ. Then there exists an open, bounded neighborhood V C M" of 6il on which J\f is 
non-vanishing. Moreover, for each x V there exist a scalar Tx > and a smooth integral 
curve ifx : {—Tx,Tx) — > M" satisfying (px{0) = x and 

d(Px , 



dt 



(t) = {Ml{^x{t)),...,Mn{iPx{t)) 
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for all t G ( ~'TxjTx)- Because the curve (fx intersects b^l transversally, there exists a scalar 
To > such that ipp is defined on (— ro,ro) for all p G bQ. After possibly rescaling M, it 
may be assumed that r = 2. 

For each x £ V define tx to be the (unique) scalar for which (p{tx,x) G bil, and note that 
\tx\ = 0{dbn{x)). It may be assumed that tx > for all x £ V HQ. Set U := {x : \tx\ < 1}. 
Then U is an open neighborhood of 60. Moreover, the flow map, (p{t,x) := ipxit), is a 
smooth map on (—1, 1) x U satisfying ip{0,x) = x as well as 

(4.1) ^it,x)=Af,i^it,x)) y ie{l,...,n}. 



C^{n) := {/ G C°°{n) \f = Oonn\U}. Then we define 21 as an operator from C^{n) to 



We can now make precise the anti-differentiation operator 21 from Section [TJ Denote by 
itself: 

2tb](x) 



J^^{goip)[s,x)ds if x G n [/, 

if xen\u. 

21 belongs to a class of operators denoted by q below; see Definition I4.4^ where the 
mapping properties are also discussed. 21 inverts J\f in the following sense. 

Lemma 4.2. Let g G C^{^)- Then 

(4.3) g{x) = 2l[Ar5](x) , x e n. 

Proof. When x G il\f/, both sides of ()4.3p equal zero. When x G Qr\U, then g ((/?(— 1, x)) = 
because ip{—l,x) G \ [/. Note that (14. ip implies 

{{Mg) o y?) (s, a;) = ^ {{g o If) {s, x)) , 

hence an application of the Fundamental Theorem of Calculus completes the proof of (|4.3p . 

□ 

4.2. The spaces yi*^ and their mapping properties. In order to organize the proof 
of Proposition 13.11 we now define several spaces of operators related to, but more general 
than 21. We group the operators both according to their form as well as according to their 
mapping properties. 

Definition 4.4. (1) Denote by Aj^Q, /i G No the space of operators acting on which 
are of the form 

J^-^ s^7^(s, x) ■ {g o ip){s, x)ds if x G H U, 



if xen\U 



(4.5) A[g]ix) = 

for some -fA G C°°{[-1,0] x U). Then A[g] G C^{n). 

(2) Denote by A^^ ,^, fi G Nq, G N, the space of operators on C^{Q) spanned by operators 
of the form A^ o for A^ G o ^"■^ < i^- 

These operators have mapping properties in weighted spaces that will be very useful 
for our purposes. We introduce the following definition: 

Definition 4.6. An operator A on C^(il) is said to belong to if there exists a constant 
C > such that 

(4.7) • A[g]\ <CJ2 ■ V £ G No, g€ C^{n). 
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Here, C does not depend on g or £. Note that because tx is defined on Q nU and both g 
and A[g] vanish on ^\U, both sides of ()4.7p are well-defined. 

The following lemma is the key to the mapping properties of the anti-derivative operator 
21 and its generalizations; in particular, it makes precise the notion of 'gaining' a factor of 
the boundary distance. As mentioned in Section [H it is a consequence of one of a group of 
inequalities due to Hardy ([l2|, sections 9.8, 9.9). 

Lemma 4.8. For given fi € Nq, define the operator B^^ on C-^(il) by 

(4.9) B [g]{x) = I ■ \9{.v{s,x)\ ds if xenpiU 

Then B^ € 

Proof. It is convenient in this proof (but not later on) to rewrite using coordinates 
(r, p) G [0, 2] X 6r2 on n y given by (r,p) ip{—T,p). Expressing B^ in these coordinates 
and changing variables gives 

(4.10) B^[g]{T,p) = J a^'\g{a,p)\da , {T,p)r^x = ip{-T,p)ennU. 

Estimate (|4.7p now follows from the following inequality of Hardy's (see |T2], Theorem 330, 
section 9.9) for / > 0: 

poo / roc \ 2 A poo 1 

(4.11) [x^ f{t)dtj dx<j^-^J^ (f+^f(^t)f dt,r>--. 
Indeed, to verify ()4.7p for B^, it suffices to observe that for j; ~ (r, p) G n C/ 

(4.12) \T'B[g]iT,p)\<T' I\n9{^,p)\ da, 

then to apply KTT\i with f{t) = f \g{t,p)\ , < i < 2, and f{t) = , t > 2, r = £, for 
p ^hVt fixed, and lastly, to integrate over hVL. We also use the equivalence, with a constant 
that depends only on and A/", of the volume elements dV and dr x d^^ on n where 
dV and di,Q_ denote the Euclidean volume elements on M" and 617, respectively. Also note 
that for r = ^ > 0, the constant on the right hand side of (j4.1ip is less than or equal to 4. 
Replacing it by 4 yields a constant in ()4.7p that does not depend on i. □ 

If A is of the form A = A^ o , then the inequality \s\ < \s\ + tx = iip{s,x) the 
boundedness of |7yi| on [—1,0] x U imply 

(4.13) \\tiA[g]\\ = \\tiA,[D^g]\\ < \\tiB,[D^ g]\\ < li^^+^D'^gll, 
where the last inequality in (I4.13P follows from Lemma 14.81 This proves: 
Lemma 4.14. A},^^ C /x, € Nq. 

We will also need notation for compositions of operators: 
Definition 4.15. (1) For a multi-index a = (ai, . . . ,0^) G Nq, £ E N; define 

■^i,o = (Ai o ■ ■ ■ o Ae : Aj e Al, 



(2) Denote by A^^ ,^, a G Ng, G N and £ G N, the space of operators on C^{Q,) spanned 
by operators of the form o for G A^^ q and |/3| < v. 

With Lemma 14.141 above, we have the following weighted mapping properties of these 
operators: 
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Lemma 4.16. A^^^^, C 55"*"'"', where \a\ = Ylj=i'^j- 

4.3. On the algebra of commutators of Al ^ and differential operators. Throughout 

this subsection, X = X]j=i^iai~ denotes a vector field with smooth coefficients on (a 

neighborhood of) Q. We collect a series of elementary lemmas that give control over various 
commutators involving the operators introduced in the previous subsection and X or D^. 
The notation +, or will be used to indicate sums of operators in the indicated spaces. 

Lemma 4.17. Let A € for some ji G Nq. Then 

(4.18) [A^] G 4,0 + 4+1,1 • 

Proof. Let g G C^(0). Then 



(4.19) {XoA)[g]{x) = y\^((X,(7^))-(<7 0(^))(5,x)ds 







+ / s^''~1A{s,x) ■ Xx{g oip){s,x)) ds. 



1 



Let ^0 be the operator in ^ o such that the first term on the right hand side of (|4.19|) 
equals — j4o[(7](2;). Then it follows that 

(4.20) [A,XM{x)=Ao[g]{x)+ j"" ^s^'^A{s,x)({Xg) o ^ - X,{g o ^)Ys,x) ds. 

Aq belongs to That the term on the right hand side of (|4.'20p given by the integral 

belongs to 4+i,i be seen as follows. Because 93(0, x) = x, dipj/dx^ = + 0{s). 
Therefore, the chain rule shows that (^Xg) oLp — Xx{g o ip)^ (s, x) is a sum of terms each of 

which is of the form s'yi3{s,x)D^g{ip{s,x)), where 7^ is smooth (depending on first deriva- 
tives of X o ip and second derivatives of <p) and |/3| = 1. □ 

Lemma 4.21. Let A^^^ G A^^ ,^ for some G Nq, G N. Then 

(4.22) [^/.,.,^] e4, + 4+i,,+i. 

Proof. By definition of A'^^ y, A^^y can be written as a linear combination of operators of 
the form A^ o D^, where ^ 4o ^^^'^ 1*^1 — ^- suffices to show (|4.22p for the latter 
operators. For that, note that 

[A^, o D^,X]= Af,,o o X - X o A^o 

= A^o [D^, X] + A^,oX oD^ - X o A^oDf^ 
= A^o[dP,X] + [A^,X]oD^ 
Since is a differential operator of order — 1, A^o [D^ .,X] belongs to ^ i;3|-i- 

Furthermore, (liT8]l yields that [A^,, X] o D^^ € 4,|/3| + 4+i l/3|+i- ° 

Lemma [4.17l extends to iterated commutators as follows. SetC^(A) = [C^~"'^(y4), X] with 
Cj.{A) = [A, X]. That is, C^iA) equals the v-Md iterated commutator [■ ■ ■ [A, X], X], ■ ■ ■ X]. 

Lemma 4.23. Let A be given as in Lemma \4.11\ Then C'^{A) G SJ=o4+ii' 

Proof. The proof is done via induction on i^. Note first that the case = 1 is Lemma 
14.171 Next suppose that C^~^{A) G Ej=d4+jj holds for some G N. It fohows from 

Lemma 1121] that [Af^+jj,X] G + 4+i+ij+i ^M+iJ ^ ^^^'^^ completes 

the proof. □ 
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We next consider commutators with higher order derivatives. 

Lemma 4.24. If A e Aj^ Q for some /x e No, then [A,DI^] £ -^l^nsi^i + 

Proof. The proof is done via induction on The case |/3| = 1 follows from Lemma |4.17[ 
Suppose now that [A, D'^] G A\ ^ -^k+i u-i holds for any multi- index a of length v — 1 
for some € N and for all k G Nq. Let /3 be a multi- index of length v. Then may be 
written as some multi- index a of length v — 1. It then follows that 

[A, L>^] = [A, D'^^''] 0" + D^~" oAoD''- D^'" o D"" o A 

= [A, D^^"] o D'' + D^-'' o [.4, L>^] . 

It follows from Lemma STT] and the fact that |/3 - cj| = 1 that [A, D^-"] o G Aj^ |^|_^ + 
A^^^-^ 1^1 . The induction hypothesis furnishes two operators Ai £ A^^ ^,_2 and A2 E ^^+1 1,-1 
so that [A, D"] = A1+A2. Then 

D^-" o [A, D"] = D^-"" o (^1 + A2) 

= Al o D^-'' + A2 o D^-"" + [D^-'^, Al] + [D^-", ^2] . 

The four terms on the right hand side are in, respectively, A^^ A^^j^i j,, A^^ y_2+-4j'i+i v-i-, 

and ^^+1 + -^^+2,i/- We have used Lemma [4.211 for the last two terms (|/3 — a\ = l!). 

Taking into account the (trivial) inclusions Aj^i^_2 C Aj^^^_i, -4.^+i^,y_i C Ajj^^^_i, and 

-^M+2,iy ^ -^^+1,1/' shows that D^-*^ o [A,D''] G + -^ji+i,!^- This completes the 

induction. □ 

For compositions of operators, Lemma 14.241 takes on the following form. 

Lemma 4.25. // Aa^i, G A^^ ^ for some multi-index a G Nq, z/ , ^ G N, then 

e 

(4.26) [Aa,.,D^] G + E'^a+e^.+ l/?!' 

where ej is the standard j-th unit vector. 

Proof. We first consider the case z/ = 0; in this case, the proof is by induction on £. The 
case £ = 1 is Lemma I4.24[ The induction step is analogous to that in the proof of Lemma 
14.241 with the multi-index a now playing the role of /3 there. We leave the details to the 
reader. 

When u > 0, A a^y is a linear combination of operators of the form A^ q o D'^ with I7I < v. 
Thus [Aa^u,D^] is a linear combination of terms of the form 

[Aafi o D^, D'^] = Ac,,o oD^ oD^ -D^ o A^.^ o = [A^fi, D^] o 

because D"' oD^ = oD'' . (|4.26|) now follows from the case v = (already shown) applied 
to[A„,o,^^]. □ 

4.4. Proof of Proposition 13.11 Let i7 and T be as in Proposition 13.11 Near T = 
a(Ti + L), with both Ti and L tangential, Ti real, L of type (1, 0), and a a smooth function 
that does not vanish near bO, (see (j2.10|) ). It is easy to see that the conclusion of Proposition 
13.11 holds for T if and only if it holds for T\ -\- L (however, the 'H^'s in (i) change and so 
do the constants Cj^k in (ii), see also the short discussion surrounding (j2.12p ). Therefore, 
it may be assume that T = Ti + L, with Ti and L as above. Set M := JTi. Because Ti is 
complex transversal, J\f is transversal to 60, and so the general set-up of ^4.11 applies. 

The proof of Proposition 13.11 will be achieved in two steps. The first step consists in 
replacing M in Lemma 14.21 by iT, adding the necessary correction, and then iterating the 
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result. The key for the proof of Proposition 13.11 is that when apphed to a holomorphic 
function, the correction term is benign, as a result of the Cauchy-Riemann equations, see 
(f4:39]l below. 

Lemma 4.27. Let h € C°°{^) for some A; € N. Let C, € CQ°($7n{7) a non-negative function 
which is identically 1 in a neighborhood of bQ contained in U. Then 

fc-i 

(4.28) iCh) (x) = (21 o T)^ [Ch] + ^ i^' (21 o T)^' o 21 o (AT - iT)[Ch]. 

j=0 

Proof. The proof is done by induction on k. Suppose first that k = 1. Then Lemma 14.21 
yields 

Ch = 2l[AA(C/i)] = 2l[iT(C/i)] + 2l[(AA - iT){Ch)] 

(4.29) = i (21 o T) [Ch] + ^o{M-iT) [Ch] . 
For the induction step suppose that 

k-2 

(4.30) Ch = (21 o rf'^ [Ch] + ^ i^ (21 o Ty o^o{M- iT)[Ch] 

j=0 

holds. Using identity (j4.29p to replace ^/i in the first term of the right hand side of (j4.30p 
gives the result. □ 

In the second step, we will write the powers (21 oT)^ in terms of compositions of the 
form (T)"* o 21 with good control of 21. This is accomplished in the following lemma; its 
proof relies heavily on the machinery of §4.21 and §4.31 

Lemma 4.31. Let A € Aq, and X a vector field with smooth coefficients on Then, for 
any £ G N, there exist operators Gf^, m £ {0, . . . ,£}, which belong to J2{u<e-m \a\-u>o} •^a,u 
such that 

e 

m=0 

Proof. The proof is again by induction on i. For £ = I, commuting A hy X yields 

AoX = X oA+[A,X]=: X oG\ + Gl. 

It follows from the definition of A that G\ = A G Aqq. Furthermore, Lemma 14.171 gives 
that Gq belongs to Aq q + A\ i- 

For the induction step suppose that 

{AoX)'-' = Y,X'^oG'-' 

m=0 

holds for some G J2{,y<e-i-m,\a\-u>o} -^i'u ■ Then 

e-i I 
{AoXf=Y,AoX^+'oG'^' = Y,A°X^°G'^-i 

m=0 m=l 

(4.32) =Y,(x^oAo Gl-\ + [A, X^] o G'^I^ 



m=l 
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Since A£ Aqq, it follows that 

A o Gt_\ € Yl <^ ■ 

{u<e-m,\a\~u>0} 

To deal with the second term on the right-hand side of (I4.32p . we use the formula 

(4.33) [A,X"^] = Y,r]x^oC'^^~\A), 

j=o ^ 

where the iterated commutators C* are defined just before Lemma 14.231 ( (I4.33P is purely 
algebraic and is easily proved by induction on m; alternatively, see Lemma 2 or (21] , 
formula (3.54).) What must be shown then, is that 

For that, recall first that Lemma [4.231 savs that C^~-' {A) € Ylli—o ^\k- Furthermore, it 
follows from Lemma 14.251 that for \(3\<k 

{v<e.-m,\a\-i>>0} 

It then follows that C^"-' {A) o G^j^\ is contained in 

{v<t~m,\a\-u>Q ,0<k<m~j} 

Set V = V + k, then v < £ — m implies that V < £ — j , since k < m — j . Moreover, setting 
a = {k, a) yields \a\ — V = \a\ — v > 0. Hence 

{u<e-j,\a\-u>0} 

which completes the proof. □ 

We are now ready to prove Proposition 13.11 This amounts to combining Lemmas 14.271 
and 14. 3H to obtain a representation (i) with Tij G C°°{Q); this works for h G C°°(0). The 

final step then consists in obtaining the required estimates (ii) and membership in H^{^}) 
when h is holomorphic. 

Proof of Proposition \3.1[ For h and C given as in Lemma 14.271 use Lemmas 14.271 and 14.311 
(with 2t and T in place of A and X, respectively) in (I4.28P to obtain that 

Ch=Y T^-^l 



m=0 



where 

(4.34) Hl:=Gl\i\hl 
and 

(4.35) := I i^^G^ + ^ i^Gi^ o 21 o (AT - iT) | [Ch] , < m < A; - 1. 

Because h € C°°(0), so are the 1-0^, < m < k. This establishes the representation (i) in 
Proposition 13.11 except for membership in H^{Q). 
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We now prove the estimates (ii). By Lemmas 14.311 and 14.161 it follows that 



njk 



< \\h\ 



where the second step follows from (12. 2p : we use here that h in Proposition 13. II is holomor- 
phic. By analogous reasoning, we have 



(4.36) 



\Gi[Ch]\\< ^ P^'^ll-fc-H ^ ll^l 



where a is such that G -^a u fo^ some u < \a\. Note that D^^h is also holomorphic, so 
that (|2.2p applies. The last inequality holds because |/3| < \a\. 

To obtain the claimed estimates for the remaining terms in 71^^ first note that because 
G I];/<j-m,|Q|-i/>o-^".!^' G:^ o 21 is a sum of terms of the form 

< o o... o Al^.o 1)7 o2l = <o...oAl^.o2lo 1)7 +^l^o...o^l^,o [1)^,21], 

where \'y\ < \a\ < < j - m < j < k - 1. Because [D'^,^ € Al ^^^_^ + A{ |^| (Lemma [i^ . 
it follows, in view of Lemma 14.141 that 

(4.37) \\Gi,o^{M-zT)[Ch]\\ < \\{M-iT)[Ch]\\^_^. 
The Cauchy-Riemann equations for h yield 

(4.38) Mh = iTih = iTih = iTh 
since Ti = Ti and Lh = 0. Consequently, 

(4.39) iM-iT)[Ch] = {{M-iTm)h. 

Since ((A/" — zr)[C]) has compact support inQnU that does not depend on h, it follows (for 
example from (j2.2p by using that factors of r are bounded away from zero on this support, 
so that introducing them will at most 'increase' the norm) that there exists a constant 
such that 

(4.40) ||(Ar-ir)[C/i]||,_i = \\{{M-iT)[C])h\\^_^ < Ck\\h\U ■ 
Combining (032]), (0311), and (lOOl) gives that 



-T/A 



< \\h\ 



for all m € {0, . . . , A; — 1}. This concludes the proof of (ii). 

It remains to see that € H''{Q). First note that because {J\f - iT)[Ch] G C^(il), it 
follows easily from the form of the operators Gm that Gm o 21 o (M — iT)[(^h] G G°^{Q) C 
H^{Q), j < m < {k — 1). The remaining contributions in (I4.34p and (|4.35p that need to be 
checked are of the form G^[Ch], < m < k. That is, we need to show that D^G^[(h] G 
L2(0) for |/3| < k. But D^^G^ = G^D^ - [G^, Z?^] . The argument is now analogous to the 
discussion above. For example, in view of Lemmas 14.311 14.251 and 14.161 the commutator 
[GJ^, L*^] is a sum of terms in + 



k+\a\ 



^ <S^+\^l with 1^ <k 



m and \a\ > v. 



Therefore, arguing as in (I4.36p . we see that the contribution of each of these terms to 
II [G^,/)'^] {Ch)\\ is dominated by Y.\'i\<y+\p\ ll-C^^II-fc-ioi- Because \^\ < v + < \a\ + 
1 ,5 1 < |a| + k, all these terms are indeed dominated by \\h\\. 

The argument for G^D^ is similar. This concludes the proof of Proposition 13.11 □ 
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5. Proof of Theorem 11.101 



Proof of Theorem \1.10l As in the proof of Theorem II. 1) we invoke duahty via Proposition 
and Remark 12.81 it suffices to show that 



(5.1) 



fg 



<C\\f\\-\\g\ 



for f G and g G A^{Q). We now use that for holomorphic functions, membership in 
A~"^{Q) for some m is equivalent to having a blow up rate near the boundary of at most a 
power of l/dbQ^z), where dhQ{z) is the boundary distance function. We have the estimates 



(5.2) 



C^\\h\\^rn-2n-2 < SUp \h{z)\ ■ dm{z) 



m+2n 



in both estimates, if the right-hand side is finite, then so is the left hand side (and the 
estimate holds; that is, the estimates are genuine estimates as opposed to a priori estimates). 
The inequalities (j5.2p are essentially Lemma 2 in [5], except that the norm in the left most 
term there is the (— m — 4n)-norm. The stronger version given here is in [20j, Theorems 1.1 
and 1.3; see in particular the proof of the implication (iii) (iv) in Theorem 1.1. 

Applying the second inequality in (j5.2p to / (for m = 0) and to g (for m = ki) yields 



sup 1/(^)1 •4n(^)'"<C2 



and snp\g{z)\-dbn{z)'''+^'' <Cl \\g\ 



Multiply these inequalities, then apply the first inequality in ()5.2p to fg (which is holomor- 
phic). The conclusion is that fg, hence fg, belongs to H~^^~'^^~'^{Q), and 

(5-3) \\7g\\_ki-An-2 - '^fcill/ll • Ibll-fci- 

On the other hand, the following estimate also holds: 



(5.4) 



fg 



fg 



<Ck, 



\f9\ 



-k-i-in-2 



HI 



ki+in+2- 



The inequality in (15. 4p is Proposition 1.9 in |20j which gives this estimate for the pairing 
of a harmonic function in some H^"^{0,) with an arbitrary function in C°°{^}). Note that 

since both / and g are in L'^{Q), fg is integrable, and the integral denoted by J in [20] 
coincides with the ordinary integral over $7. Combining (j5.3p and (j5.4p completes the proof 

□ 



of Theorem 11.101 



Remark 5.5. When Condition R holds, one can extend B by duality to a projection B from 
the dual (C°°(r2)) into U^^^^^($7). This was observed in [14], in a note added in proof 
(where the idea is attributed to Nirenberg and Treves). The conclusion of Theorem 11.101 

then Bf is smooth 



remains true for this extended projection: when / is in U 
up to the boundary (see [20], Theorem 3.4 
(C°°(0))*). A similar discussion applies when (|1.2p holds 



oo A- 



for the 'canonical' inclusion U^^^A ^{^l) ^ 
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